1. Field of the Invention
The present invention relates to image identification apparatus and methods of identifying images. More particularly the present invention relates to apparatus and methods for identifying images from hand drawn representations of the images.
2. Description of the Prior Art
There are many ways in which images may be created. Conventionally, an author or artist may draw a representation of the image using any appropriate means such as pencils, pens or other drawing equipment. An artist may also produce an image using a computer running a drawing application program which provides a facility for creating images and pictures by selecting pre-defined elements Such as boxes, lines and circles which may be positioned, sized and shaped using a computer mouse. Furthermore to reduce a time required to generate an image, the computer may be provided with a library of pre-generated images which may be selected and copied into a picture to produce a desired scene. The images may be for example parts of or complete pictures, designs for articles or representations of a scene of a play, television programme or film.
In the field of character recognition, images representing characters from an alphabet are recognised and the character from this alphabet is generated in response to input representations of those characters. An example of this is provided in a disclosure entitled xe2x80x9cNeural Network Classifiers for Optical Chinese Character Recognitionxe2x80x9d by R. Romero, R. Berger, R. Thibadeau and D. Touretsky of the Imaging Systems Lab., Robotics Institute, Carnegie Mellon University published on the Internet at http://www-cgi.cs.cmu.edu. Furthermore the input representations may be hand drawn. In this case to assist in identifying the image, the hand drawn representation may be segmented into a number of different strokes of a pen which were performed in drawing the representation. This is disclosed in an article entitled xe2x80x9cPen-tip velocity in Handwritingxe2x80x9d, published on the Internet at http://hwr.nici.kun.nl/recog/hwr-tutor/veocity.html, copyright Lambert Schomaker, Apr. 1, 1996.
Generally it is desirable to provide a facility for generating an image from a hand drawn representation of the image in an efficient manner. This is appropriate, for example, during early stages of expressing ideas and initial thoughts for a picture, design or scene, where creative ideas are readily expressed as hand drawn representations.
According to the present invention there is provided an image identification apparatus for identifying an image from a hand drawn representation of at least part of the image, the image identification apparatus comprising an image processor arranged in operation to generate a reference identification in response to spatial samples produced from at least part of the hand drawn representation, the reference identification being indicative of a first estimate of which of a plurality of pre-stored images corresponds to the hand drawn representation, and a controller which is arranged in operation to cause the image processor to produce a refined reference identification from the spatial samples and further spatial samples produced from a further part of the hand drawn representation, the refined reference identification being indicative of a refined estimate of which of the plurality of the pre-stored images corresponds to the hand drawn representation.
It is possible to make an estimation of the identity of an image from a hand drawn representation of part of the image. As such by generating a reference identification indicative of a first estimate of which of a plurality of pre-stored images corresponds to the hand drawn representation, which may not be complete, and generating a refined reference identification following further spatial samples corresponding to further parts of the hand drawn representation of the image, a refined estimate of the identity of the hand drawn representation from the plurality of pre-stored images may be made. The control processor is arranged to operate with the image processor to generate further refined estimates of the identification of the image from subsequent parts of the hand drawn representation of the image. This provides a particular advantage in improving the efficiency with which an image may be produced from a hand drawn representation of the image in that by generating a first estimate of the image from the spatial samples produced response to the first part of the hand drawn representation, a first estimate of the image may be produced which may or may not correspond to the image desired. Therefore by continuing to draw a further part or parts of the image, or indeed revising a part already drawn, from which farther spatial samples are produced and generating the refined reference identification from an accumulation of the spatial samples and the further spatial samples, the refined estimate of which of the plurality of pre-stored images corresponds to the desired image may be produced from the more complete hand drawn representation or the image. This provides an improvement in a time taken to produce an image because a complete image may be formed from the pre-stored images, from a hand drawn representation of only part of the image. This is particularly advantageous for generating a scene which is made up of a plurality of the pre-stored images.
Although the spatial samples may be generated before being fed to the image identification apparatus, the image identification apparatus may preferably have a data store coupled to the image processor and to the control processor which serves to store at least one of the spatial samples and the further spatial samples. The control processor may be arranged in operation to feed at least one of the spatial samples and the further spatial samples to the image processor to generate the refined reference indication. The data store therefore stores the spatial samples corresponding to a particular hand drawn representation of the image. Further samples are then added to the data store as further parts of the image are drawn. In this way, spatial samples corresponding to the hand drawn representation of the image may be built up in the data store until enough samples are present to enable the correct image to be identified.
In a preferred embodiment the control processor may be arranged to receive a trigger signal in response to the part of the hand drawn representation being generated and the further part of the hand drawn representation being generated, the control processor being arranged in operation to communicate at least one of the spatial samples and the further spatial samples to the image processor in response to the trigger signal, so as to enable the refined reference identification to be generated. Although the image identification apparatus may operate continuously in that the apparatus may be arranged to continuously update the estimate of the image corresponding to the hand drawn representation of the image, by providing a trigger signal indicative of a completion of at least part of the hand drawn representation of the image, the control processor may generate an estimate of the image after a user has completed part of the image and from which the user would consider that the identification apparatus could sufficiently identify and distinguish the thus far hand drawn representation of the image from the plurality of pre-stored images.
Although the spatial samples corresponding to the hand drawn representation of the image may be pre-generated and stored before being fed to the image identification apparatus, advantageously the image identification apparatus may comprise a drawing processor having a drawing tool for use in creating the hand drawn representation of the image, the sequence of spatial samples being generated in accordance with the movement of the drawing tool. Furthermore the trigger signal indicative of completion of at least part of the hand drawn representation of the image may be provided by the drawing processor. In a preferred embodiment the drawing tool may be a pen means and a drawing surface, the trigger signal being generated by the drawing processor when the pen is substantially removed from the drawing surface. As such the image identification apparatus is provided with a particular improvement in that the user can efficiently add further parts of the hand drawn representation of the image, when the spatial samples so far produced are not sufficient to identify the image. In this way sufficient spatial samples may be established from which the image identification apparatus can identify the image. The user may then lift the pen from the drawing surface which is taken as the trigger signal that the image processing apparatus should form the next estimate corresponding to the refined reference identification of which of the plurality of the pre-stored images corresponds to the hand drawn representation of the image.
As an alternative to providing a trigger signal externally, the image identification apparatus may have a clock coupled to the control processor which serves to provide a temporal reference from which a predetermined time lapse from when the spatial samples and the further spatial samples were produced can be measured. The control processor may be arranged in combination with the image processor to generate at least one of the reference identification and the refined reference identification in response to the lapse of the predetermined time period. This alternative means for generating the reference identification and the refined reference identification therefore does not require a trigger signal and so the image corresponding to the spatial samples may be generated automatically without further user input.
In a preferred embodiment the image processor may comprise a segment processor arranged in operation to determine from the spatial samples or the further spatial samples stroke data representative of strokes performed in drawing the hand drawn representation of the image, a stroke pre-processor arranged in operation to generate parameter data by associating the sequence of spatial samples with the determined stroke data of the hand drawn representation and an image identifier coupled to the stroke pre-processor arranged in operation to generate the reference identification or the refined reference identification from the parameter data. Similarly in a preferred embodiment the image identification apparatus may comprise a visual display means coupled to the control processor and the data store may be arranged to store the plurality of pre-stored images. The control processor may be arranged in operation to retrieve from the data store image data representative of the image which corresponds to the hand drawn representation in accordance with at least one of the reference identification and the refined reference identification, and to display the image data on the visual display means.
As already explained, the trigger signal may be generated when a user lifts the pen from the drawing surface of the drawing tool. However in a further improvement, the image identification apparatus may further comprise a user interface device coupled to the control processor and arranged in operation to provide a user indication as to whether or not the corresponding image is in accordance with the hand drawn representation desired by the user. The corresponding image may then be saved in response to the user indication. The user interface device therefore provides a means to indicate to the control processor of the image identification apparatus that the image displayed on the visual display means is that which corresponds to the desired hand drawn representation of this image. In a further improvement in response to the user indication, the data store may be reset to receive spatial samples representative of a hand drawn representation of a further image. Therefore once the user indication has been received which indicates that the spatial samples and the further spatial samples which may have been added to complete further the hand drawn representation of the image and from which the estimate of the image is generated, these spatial samples are removed from the data store so that further spatial samples may be received which correspond to a further desired image. The user interface device may be for example a computer mouse, a key board or indeed may be provided by the drawing tool or any similar device.
According to an aspect of the present invention there is provided a method of identifying an image from a hand drawn representation of at least part of the image, the method comprising the steps of generating a reference identification from spatial samples produced in response to at least part of the hand drawn representation of the image, which reference indication is indicative of a first estimate of which of a plurality of pre-stored images corresponds to the hand drawn representation, generating a refined reference identification from the spatial samples and further spatial samples produced from a further part of the hand drawn representation, the refined reference identification being indicative of a refined estimate of which of the plurality of prestored images corresponds to the hand drawn representation and the further parts of the hand drawn representation.
Further aspects and features of the present invention are provided and illustrated with reference to the appended claims.